{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6qJus9tz95HJ"
   },
   "source": [
    "# Understanding Retrieval Question Answering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ABLFlOy795HJ"
   },
   "source": [
    "### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XxOaMfHJ95HK",
    "outputId": "6123adf2-1d26-473a-8ec7-8b7cdc4e681a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -Uqqq rich openai tiktoken wandb langchain unstructured tabulate pdf2image chromadb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "Iein9J7K95HK"
   },
   "outputs": [],
   "source": [
    "import os, random\n",
    "from pathlib import Path\n",
    "import tiktoken\n",
    "from getpass import getpass\n",
    "from rich.markdown import Markdown\n",
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J44eDUkH95HL"
   },
   "source": [
    "You will need an OpenAI API key to run this notebook. You can get one [here](https://platform.openai.com/account/api-keys)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vh_X-kEv95HL",
    "outputId": "a2821b81-249c-46bf-9929-1d47937770ca"
   },
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      " ········\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API key configured\n"
     ]
    }
   ],
   "source": [
    "if os.getenv(\"OPENAI_API_KEY\") is None:\n",
    "  if any(['VSCODE' in x for x in os.environ.keys()]):\n",
    "    print('Please enter password in the VS Code prompt at the top of your VS Code window!')\n",
    "  os.environ[\"OPENAI_API_KEY\"] = getpass(\"\")\n",
    "\n",
    "assert os.getenv(\"OPENAI_API_KEY\", \"\").startswith(\"sk-\"), \"This doesn't look like a valid OpenAI API key\"\n",
    "print(\"OpenAI API key configured\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tAvdRw9H95HM"
   },
   "source": [
    "## Langchain\n",
    "\n",
    "[LangChain](https://docs.langchain.com/docs/) is a framework for developing applications powered by language models. We will use some of its features in the code below. Let's start by configuring W&B tracing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "id": "KeVsMGoI95HM"
   },
   "outputs": [],
   "source": [
    "# we need a single line of code to start tracing langchain with W&B\n",
    "os.environ[\"LANGCHAIN_WANDB_TRACING\"] = \"true\"\n",
    "\n",
    "# wandb documentation to configure wandb using env variables\n",
    "# https://docs.wandb.ai/guides/track/advanced/environment-variables\n",
    "# here we are configuring the wandb project name\n",
    "os.environ[\"WANDB_PROJECT\"] = \"maven-article\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3Kjb4VCe95HM"
   },
   "source": [
    "## Parsing documents\n",
    "\n",
    "We will use a small sample of markdown documents in this notebook. Let's find them and make sure we can stuff them into the prompt. That means they may need to be chunked and not exceed some number of tokens. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "id": "GdnorDBV95HN"
   },
   "outputs": [],
   "source": [
    "MODEL_NAME = \"text-davinci-003\"\n",
    "# MODEL_NAME = \"gpt-4\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YNjbTNro95HN",
    "outputId": "5af6d794-8f3b-4a18-cd3d-25b5faeb2635"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.document_loaders import DirectoryLoader\n",
    "\n",
    "def find_md_files(directory):\n",
    "    \"Find all markdown files in a directory and return a LangChain Document\"\n",
    "    dl = DirectoryLoader(directory, \"**/*.txt\")\n",
    "    return dl.load()\n",
    "\n",
    "documents = find_md_files('article_data')\n",
    "len(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "q46oTWl8-yQ3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='I would like to get your all  thoughts on the bond yield increase this week.  I am not worried about the market downturn but the sudden increase in yields. On 2/16 the 10 year bonds yields increased by almost  9 percent and on 2/19 the yield increased by almost 5 percent.\\n\\nKey Points from the CNBC Article:\\n\\n**The “taper tantrum” in 2013 was a sudden spike in Treasury yields due to market panic after the Federal Reserve announced that it would begin tapering its quantitative easing program.**\\n\\n**Major central banks around the world have cut interest rates to historic lows and launched unprecedented quantities of asset purchases in a bid to shore up the economy throughout the pandemic.**\\n\\n**However, the recent rise in yields suggests that some investors are starting to anticipate a tightening of policy sooner than anticipated to accommodate a potential rise in inflation.**\\n\\nThe recent rise in bond yields and U.S. inflation expectations has some investors wary that a repeat of the 2013 “taper tantrum” could be on the horizon.\\n\\nThe benchmark U.S. 10-year Treasury note climbed above 1.3% for the first time since February 2020 earlier this week, while the 30-year bond also hit its highest level for a year. Yields move inversely to bond prices.\\n\\nYields tend to rise in lockstep with inflation expectations, which have reached their highest levels in a decade in the U.S., powered by increased prospects of a large fiscal stimulus package, progress on vaccine rollouts and pent-up consumer demand.\\n\\nThe “taper tantrum” in 2013 was a sudden spike in Treasury yields due to market panic after the Federal Reserve announced that it would begin tapering its quantitative easing program.\\n\\nMajor central banks around the world have cut interest rates to historic lows and launched unprecedented quantities of asset purchases in a bid to shore up the economy throughout the pandemic. The Fed and others have maintained supportive tones in recent policy meetings, vowing to keep financial conditions loose as the global economy looks to emerge from the Covid-19 pandemic.\\n\\nHowever, the recent rise in yields suggests that some investors are starting to anticipate a tightening of policy sooner than anticipated to accommodate a potential rise in inflation.\\n\\nWith central bank support removed, bonds usually fall in price which sends yields higher. This can also spill over into stock markets as higher interest rates means more debt servicing for firms, causing traders to reassess the investing environment.\\n\\n“The supportive stance from policymakers will likely remain in place until the vaccines have paved a way to some return to normality,” said Shane Balkham, chief investment officer at Beaufort Investment, in a research note this week.\\n\\n“However, there will be a risk of another ‘taper tantrum’ similar to the one we witnessed in 2013, and this is our main focus for 2021,” Balkham projected, should policymakers begin to unwind this stimulus.\\n\\nLong-term bond yields in Japan and Europe followed U.S. Treasurys higher toward the end of the week as bondholders shifted their portfolios.\\n\\n“The fear is that these assets are priced to perfection when the ECB and Fed might eventually taper,” said Sebastien Galy, senior macro strategist at Nordea Asset Management, in a research note entitled “Little taper tantrum.”\\n\\n“The odds of tapering are helped in the United States by better retail sales after four months of disappointment and the expectation of large issuance from the $1.9 trillion fiscal package.”\\n\\nGaly suggested the Fed would likely extend the duration on its asset purchases, moderating the upward momentum in inflation.\\n\\n“Equity markets have reacted negatively to higher yield as it offers an alternative to the dividend yield and a higher discount to long-term cash flows, making them focus more on medium-term growth such as cyclicals” he said. Cyclicals are stocks whose performance tends to align with economic cycles.\\n\\nGaly expects this process to be more marked in the second half of the year when economic growth picks up, increasing the potential for tapering.\\n\\n## Tapering in the U.S., but not Europe\\n\\nAllianz CEO Oliver Bäte told CNBC on Friday that there was a geographical divergence in how the German insurer is thinking about the prospect of interest rate hikes.\\n\\n“One is Europe, where we continue to have financial repression, where the ECB continues to buy up to the max in order to minimize spreads between the north and the south — the strong balance sheets and the weak ones — and at some point somebody will have to pay the price for that, but in the short term I don’t see any spike in interest rates,” Bäte said, adding that the situation is different stateside.\\n\\n“Because of the massive programs that have happened, the stimulus that is happening, the dollar being the world’s reserve currency, there is clearly a trend to stoke inflation and it is going to come. Again, I don’t know when and how, but the interest rates have been steepening and they should be steepening further.”\\n\\n## Rising yields a ‘normal feature’\\n\\nHowever, not all analysts are convinced that the rise in bond yields is material for markets. In a note Friday, Barclays Head of European Equity Strategy Emmanuel Cau suggested that rising bond yields were overdue, as they had been lagging the improving macroeconomic outlook for the second half of 2021, and said they were a “normal feature” of economic recovery.\\n\\n“With the key drivers of inflation pointing up, the prospect of even more fiscal stimulus in the U.S. and pent up demand propelled by high excess savings, it seems right for bond yields to catch-up with other more advanced reflation trades,” Cau said, adding that central banks remain “firmly on hold” given the balance of risks.\\n\\nHe argued that the steepening yield curve is “typical at the early stages of the cycle,” and that so long as vaccine rollouts are successful, growth continues to tick upward and central banks remain cautious, reflationary moves across asset classes look “justified” and equities should be able to withstand higher rates.\\n\\n“Of course, after the strong move of the last few weeks, equities could mark a pause as many sectors that have rallied with yields look overbought, like commodities and banks,” Cau said.\\n\\n“But at this stage, we think rising yields are more a confirmation of the equity bull market than a threat, so dips should continue to be bought', metadata={'source': 'article_data/article.txt'})]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "NFlSYyqs95HN"
   },
   "outputs": [],
   "source": [
    "# We will need to count tokens in the documents, and for that we need the tokenizer\n",
    "tokenizer = tiktoken.encoding_for_model(MODEL_NAME)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "E4KaCl_395HN",
    "outputId": "f24d3049-9a96-4ab7-a64f-b74c315ca7cc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1414]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# function to count the number of tokens in each document\n",
    "def count_tokens(documents):\n",
    "    token_counts = [len(tokenizer.encode(document.page_content)) for document in documents]\n",
    "    return token_counts\n",
    "\n",
    "count_tokens(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U_tyjspX95HN"
   },
   "source": [
    "We will use `LangChain` built in `MarkdownTextSplitter` to split the documents into sections. Actually splitting `Markdown` without breaking syntax is not that easy. This splitter strips out syntax.\n",
    "- We can pass the `chunk_size` param and avoid lenghty chunks.\n",
    "- The `chunk_overlap` param is useful so you don't cut sentences randomly. This is less necessary with `Markdown`\n",
    "\n",
    "The `MarkdownTextSplitter` also takes care of removing double line breaks and save us some tokens that way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hPlKZpk595HN",
    "outputId": "3cbfbf48-2d02-4fc4-97ec-87d4a8868ffb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9, 220)"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.text_splitter import MarkdownTextSplitter\n",
    "\n",
    "md_text_splitter = MarkdownTextSplitter(chunk_size=1000)\n",
    "document_sections = md_text_splitter.split_documents(documents)\n",
    "len(document_sections), max(count_tokens(document_sections))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IuYffsl195HO"
   },
   "source": [
    "let's look at the first section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 166
    },
    "id": "FIepHKfr95HO",
    "outputId": "e9e70017-de7e-4916-c50e-ac35adbe8688"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">I would like to get your all  thoughts on the bond yield increase this week.  I am not worried about the market    \n",
       "downturn but the sudden increase in yields. On 2/16 the 10 year bonds yields increased by almost  9 percent and on \n",
       "2/19 the yield increased by almost 5 percent.                                                                      \n",
       "\n",
       "Key Points from the CNBC Article:                                                                                  \n",
       "\n",
       "<span style=\"font-weight: bold\">The “taper tantrum” in 2013 was a sudden spike in Treasury yields due to market panic after the Federal Reserve </span>   \n",
       "<span style=\"font-weight: bold\">announced that it would begin tapering its quantitative easing program.</span>                                            \n",
       "\n",
       "<span style=\"font-weight: bold\">Major central banks around the world have cut interest rates to historic lows and launched unprecedented quantities</span>\n",
       "<span style=\"font-weight: bold\">of asset purchases in a bid to shore up the economy throughout the pandemic.</span>                                       \n",
       "\n",
       "<span style=\"font-weight: bold\">However, the recent rise in yields suggests that some investors are starting to anticipate a tightening of policy </span> \n",
       "<span style=\"font-weight: bold\">sooner than anticipated to accommodate a potential rise in inflation.</span>                                              \n",
       "</pre>\n"
      ],
      "text/plain": [
       "I would like to get your all  thoughts on the bond yield increase this week.  I am not worried about the market    \n",
       "downturn but the sudden increase in yields. On 2/16 the 10 year bonds yields increased by almost  9 percent and on \n",
       "2/19 the yield increased by almost 5 percent.                                                                      \n",
       "\n",
       "Key Points from the CNBC Article:                                                                                  \n",
       "\n",
       "\u001b[1mThe “taper tantrum” in 2013 was a sudden spike in Treasury yields due to market panic after the Federal Reserve \u001b[0m   \n",
       "\u001b[1mannounced that it would begin tapering its quantitative easing program.\u001b[0m                                            \n",
       "\n",
       "\u001b[1mMajor central banks around the world have cut interest rates to historic lows and launched unprecedented quantities\u001b[0m\n",
       "\u001b[1mof asset purchases in a bid to shore up the economy throughout the pandemic.\u001b[0m                                       \n",
       "\n",
       "\u001b[1mHowever, the recent rise in yields suggests that some investors are starting to anticipate a tightening of policy \u001b[0m \n",
       "\u001b[1msooner than anticipated to accommodate a potential rise in inflation.\u001b[0m                                              \n"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Markdown(document_sections[0].page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fl1g_5Rq95HO"
   },
   "source": [
    "## Embeddings\n",
    "\n",
    "Let's now use embeddings with a vector database retriever to find relevant documents for a query. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "id": "7FZppvnB95HO"
   },
   "outputs": [],
   "source": [
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.vectorstores import Chroma\n",
    "\n",
    "# We will use the OpenAIEmbeddings to embed the text, and Chroma to store the vectors\n",
    "embeddings = OpenAIEmbeddings()\n",
    "db = Chroma.from_documents(document_sections, embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lrMaSJRk95HO"
   },
   "source": [
    "We can create a retriever from the db now, we can pass the `k` param to get the most relevant sections from the similarity search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "id": "UIlC_wvl95HO"
   },
   "outputs": [],
   "source": [
    "retriever = db.as_retriever(search_kwargs=dict(k=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "['What is the \"taper tantrum\" and how does it relate to the recent increase in bond yields?',\n",
    " 'How have major central banks responded to the pandemic and what impact has this had on bond yields?',\n",
    " 'What factors have contributed to the recent rise in U.S. inflation expectations?',\n",
    " 'How do rising bond yields typically affect stock markets?',\n",
    " 'What is the risk of another \"taper tantrum\" in 2021 and why?',\n",
    " 'How have long-term bond yields in Japan and Europe been affected by the increase in U.S. Treasury yields?',\n",
    " 'What is the difference in how Europe and the United States are approaching the prospect of interest rate hikes?',\n",
    " 'Why do some analysts believe that the rise in bond yields is a \"normal feature\" of economic recovery?',\n",
    " 'How might successful vaccine rollouts and continued growth impact reflationary moves across asset classes?',\n",
    " 'What is the potential impact of overbought sectors like commodities and banks on the equity bull market?']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "id": "H047DUNQ95HO"
   },
   "outputs": [],
   "source": [
    "query = \"How do rising bond yields typically affect stock markets?\"\n",
    "docs = retriever.get_relevant_documents(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wUunwPVG95HO",
    "outputId": "63295603-7c75-4d3d-ee61-f053e4d6172f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "article_data/article.txt\n",
      "article_data/article.txt\n",
      "article_data/article.txt\n"
     ]
    }
   ],
   "source": [
    "# Let's see the results\n",
    "for doc in docs:\n",
    "    print(doc.metadata[\"source\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pKRfDCxI95HP"
   },
   "source": [
    "## Stuff Prompt\n",
    "\n",
    "We'll now take the content of the retrieved documents, stuff them into prompt template along with the query, and pass into an LLM to obtain the answer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "id": "1mL58CTo95HP"
   },
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = \"\"\"Use the following pieces of context to answer the question at the end.\n",
    "If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "Helpful Answer:\"\"\"\n",
    "PROMPT = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "context = \"\\n\\n\".join([doc.page_content for doc in docs])\n",
    "prompt = PROMPT.format(context=context, question=query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T_X_aSHY95HP"
   },
   "source": [
    "Use langchain to call openai chat API with the question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 254
    },
    "id": "IHE6106c95HP",
    "outputId": "3da2de98-8403-46b2-ffd5-2e25b1b53ec3"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "Changes to your `wandb` environment variables will be ignored because your `wandb` session has already started. For more information on how to modify your settings with `wandb.init()` arguments, please refer to <a href='https://wandb.me/wandb-init' target=\"_blank\">the W&B docs</a>."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: Streaming LangChain activity to W&B at https://wandb.ai/tripsbya/llmapps/runs/yp9m20zl\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: `WandbTracer` is currently in beta.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Please report any issues to https://github.com/wandb/wandb/issues with the tag `langchain`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Rising yields usually cause bonds to fall in price which sends yields higher. This can also spill over into stock  \n",
       "markets as higher interest rates means more debt servicing for firms, causing traders to reassess the investing    \n",
       "environment.                                                                                                       \n",
       "</pre>\n"
      ],
      "text/plain": [
       "Rising yields usually cause bonds to fall in price which sends yields higher. This can also spill over into stock  \n",
       "markets as higher interest rates means more debt servicing for firms, causing traders to reassess the investing    \n",
       "environment.                                                                                                       \n"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.llms import OpenAI\n",
    "#d24951b9a8ae9f9787c6c5f0e97d4888b41a5f4b\n",
    "llm = OpenAI()\n",
    "response = llm.predict(prompt)\n",
    "Markdown(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c8Gk0K7C95HP"
   },
   "source": [
    "## Using Langchain\n",
    "\n",
    "Langchain gives us tools to do this efficiently in few lines of code. Let's do the same using `RetrievalQA` chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 166
    },
    "id": "YnekUGyQ95HP",
    "outputId": "38f513fe-8191-4444-bc31-a49d62651e71"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">The taper tantrum was a sudden spike in Treasury yields in 2013 due to market panic after the Federal Reserve      \n",
       "announced that it would begin tapering its quantitative easing program. This was a tightening of policy after a    \n",
       "period of looser monetary policy. Similarly, the recent increase in bond yields suggests that some investors are   \n",
       "anticipating a tightening of policy sooner than anticipated due to a potential rise in inflation.                  \n",
       "</pre>\n"
      ],
      "text/plain": [
       "The taper tantrum was a sudden spike in Treasury yields in 2013 due to market panic after the Federal Reserve      \n",
       "announced that it would begin tapering its quantitative easing program. This was a tightening of policy after a    \n",
       "period of looser monetary policy. Similarly, the recent increase in bond yields suggests that some investors are   \n",
       "anticipating a tightening of policy sooner than anticipated due to a potential rise in inflation.                  \n"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "from langchain.chains import RetrievalQAWithSourcesChain\n",
    "\n",
    "qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type=\"stuff\", retriever=retriever)\n",
    "                                                  \n",
    "result = qa.run(query)\n",
    "\n",
    "Markdown(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "dabHHWQ_95HP"
   },
   "outputs": [],
   "source": [
    "import wandb\n",
    "wandb.finish()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4PO9QdmA95HP"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "environment": {
   "kernel": "python3",
   "name": "pytorch-gpu.1-13.m108",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/pytorch-gpu.1-13:m108"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
